Energy and Mean-Payoff Parity Markov Decision Processes
نویسندگان
چکیده
We consider Markov Decision Processes (MDPs) with mean-payoff parity and energy parity objectives. In system design, the parity objective is used to encode ω-regular specifications, and the mean-payoff and energy objectives can be used to model quantitative resource constraints. The energy condition requires that the resource level never drops below 0, and the mean-payoff condition requires that the limit-average value of the resource consumption is within a threshold. While these two (energy and mean-payoff) classical conditions are equivalent for two-player games, we show that they differ for MDPs. We show that the problem of deciding whether a state is almost-sure winning (i.e., winning with probability 1) in energy parity MDPs is in NP ∩ coNP, while for meanpayoff parity MDPs, the problem is solvable in polynomial time, improving a recent PSPACE bound.
منابع مشابه
Games and Markov Decision Processes with Mean-Payoff Parity and Energy Parity Objectives
In this paper we survey results of two-player games on graphs and Markov decision processes with parity, mean-payoff and energy objectives, and the combination of mean-payoff and energy objectives with parity objectives. These problems have applications in verification and synthesis of reactive systems in resource-constrained environments.
متن کاملA Class of Markov Decision Processes with Pure and Stationary Optimal Strategies
We are interested in the existence of pure and stationary optimal strategies in Markov decision processes. We restrict to Markov decision processes with finitely many states and actions and infinite duration. In a Markov decision process, each state is labelled by an immediate payoff and each infinite history generates a stream of immediate payoffs. The final payoff associated with an infinite ...
متن کاملApplying Blackwell optimality: priority mean-payoff games as limits of multi-discounted games
We define and examine priority mean-payoff games — a natural extension of parity games. By adapting the notion of Blackwell optimality borrowed from the theory of Markov decision processes we show that priority mean-payoff games can be seen as a limit of special multi-discounted games.
متن کاملPure Stationary Optimal Strategies in Markov Decision Processes
Markov decision processes (MDPs) are controllable discrete event systems with stochastic transitions. Performances of an MDP are evaluated by a payoff function. The controller of the MDP seeks to optimize those performances, using optimal strategies. There exists various ways of measuring performances, i.e. various classes of payoff functions. For example, average performances can be evaluated ...
متن کاملOptimizing the Expected Mean Payoff in Energy Markov Decision Processes
Energy Markov Decision Processes (EMDPs) are finite-state Markov decision processes where each transition is assigned an integer counter update and a rational payoff. An EMDP configuration is a pair s(n), where s is a control state and n is the current counter value. The configurations are changed by performing transitions in the standard way. We consider the problem of computing a safe strateg...
متن کامل